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Dear Sir: 

Applicants thank the Office for the courtesy of an interview held on Thursday, 
November 1 8, 2004. The participants discussed a proposed amendment, now submitted 
herewith, to address the objections to and rejections of the claims, e.g. under section 101, 
1 12 and 103 (in view of Mizuno and Tsubaka). The amendment to the specification and 
claims to set out the meaning of "iid" was added to the amendment at the suggestion of 
the Office to clarify things a bit. At that point it was agreed that the amendments in the 
draft and those discussed didn't/wouldn't add new matter and should place the claims in 
condition for allowance. Again, applicants thank me Exaniiner and his Supervisor for 
their handling of the case and the interview. 
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Sir 

Transmitted herewith Is a response to an office action in the subject application. 

□ Small entity status is claimed for this application under 37 CFR 1.27. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re Application of: 
Alex Gammerman 

Art Unit: 2121 

Application No. 09/831,262 

Examiner: BELL, MELITN 

Filed: June 29, 2001 

For: DATA CLASSIFICATION 

APPARATUS AND METHOD 
THEREOF 

RESPONSE TO OFFICE ACTION 

Mail Stop AF 
Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Dear Sir: 

In response to the Office Action dated May 26, 2004, please enter the folio-wing 
amendments and consider the following remarks. 



CERTIFICATE OF MAILING OR TRANSMISSION UNDER 37 CFR 1.8 

I hereby certify that this Response 10 Office Action and all accompanying documents are. on the date Indicated below, El being deposited 
with the Unlisd States Postal Service as first class mail in an envelope addressed to: Mail Stop AF, Commissioner for Patents, P.O. Box 
1450, Alexandria, VA 22313-14S0, and also a courtesy copy is C3 being ftfisimfle transmitted to the U.S. Parent and Trademark Office, 
Attention: Examiner Bell. Art Unit212l, Facsimile Number 571 -273-3680. 



Name (Print/Type) 



Signtmirt 



Susan Matz 



Date 



November 18, 2004 
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In re Appln. of:Gammerman et al 
Application No. 09/83 1 ,262 



AMENDMENTS TO THE SPECIFICATION 
Please amend the paragraph on page two starting at line 22 to read as follows: 

The present invention provides a new data classification apparatus and method 
llZT T5 classification problems and Sat provides Svtsal 

measure of confidence, valid under die iid ( independently „nH ^ticllW Hi^w^ 

assumption, for each individual classification prediction made by the new data 

classification apparatus and method. 
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AMENDMENTS TO THE CLAIMS 

10, AsPMata classification apparatus comprising: 

an input device for receiving a plurality of training classified examples and at least 
one unclassified example; 

a memory for storing said classified and unclassified examples; 

an output terminal for outputting a predicted classification for said at least one 
unclassified example; and 

a processor for identifying the predicted classification of said at least one 
unclassified example 

wherein the processor includes: 

classification allocation means for allocating potential classifications to each said 
unclassified example and for generating a plurality of classification sets, each said 
classification set containing said plurality^ of training classified examples with their 
classification and said at least one unclassified example tfjdXw ith its said allocated 
potential classification; 

assay means including an example valuation device which determines individual 
strangeness values £a^for each said training classified example fi=1.2...fl and said at 
least one unclassified'example fz'=M-l^ having an allocated potential classification^, the 
assay means determining a single Tan overall] strangeness value £d££!valid under the 
[iid] independently and identically distributed assumption for each said classification set 
in dependence on said individual strangeness values (a,) of each example bv the formula 

a comparative device for selecting the classification set to which the most likely 
allocated potential classification for said at least one unclassified example belongs, 
wherein said predicted classification output by the output terminal is said most likely 
| allocated classification according to said sjnjlejoverall] strangeness values assigned by 
said assay means; and 

a strength of prediction monitoring device for determining a confidence value for 
| said predicted classification on the basis of said single roveralll strangeness value 
assigned by said assay means to one of said classification sets to which the second most 
likely allocated potential classification of said at least one unclassified example belongs. 

11. (Canceled) 

| 12. A.[Dldata classification apparatus as claimed in claim 10, wherein Lagrange 
multipliers are used to determine said individual strangeness values. 

13. (Canceled) 

\ 
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I 14, A,[D]data classification apparatus comprising: 

an input device for receiving a plurality of training classified examples and at least 
one unclassified example; 

a memory for storing said classified and unclassified examples; 

stored programs including an example classification program; 

an oulput terminal for outputting a predicted classification for said at least one 
unclassified example; and 

a processor controlled by said stored programs for identifying the predicted 
classification of said at least one unclassified example, wherein said processor includes: 

classification allocation means for allocating potential classifications to each said 
unclassified example and for generating a plurality of classification sets, each said 
classification set containing said plurality £Z^of training classified examples with their 
classification and said at least one unclassified example ££+JLLwith its allocated potential 
classification; 

assay means including an example valuation device which determines individual 
strangeness values {q^jfor each said training classified example fy=^.../) ,and said at 
least one unclassified example (/=/+n h aving an allocated potential classification^^ the 
assay means determining q ^ , np|p [ an overall] strangeness value idhYi v alid under the 
[iid] ind»sod^ distributed . assumption for each said classification set 

in dependence on said individual strangeness values (ai) of each exampl^ALthe formula 

d{y) = t /:g / &g w) .wtow.2....f.M: 

* 4~ 1 

a comparative device for selecting the classification set to vvhich the most likely 
allocated potential classification for said at least one unclassified example belongs, 
wherein the predicted classification output by said output terminal is the most likely 
| allocated potential classification according to said fijQgjt[overall] strangeness values 
assigned by said assay means and 

a strength of prediction monitoring device for determining a confidence value for 
| said predicted classification on the basis of said single^overall] strangeness value 
assigned by said assay means to one of said classification sets to which the second most 
likely allocated potential classification of said at least one unclassified example belongs. 

| 15. A coT flpyter-imnlemented data classification method comprising: 

inputting a plurality of training classified examples and at least one unclassified 
example; 

identifying a predicted classification of said at least one unclassified example 
which includes, 

allocating potential classifications to each said unclassified example; 
| generating a plurality fl) of classification sets, each said classification set 

containing said plurality of training classified examples with their classification and said 
at least one unclassified example (7+1) w ith its allocated potential classification; 

determining an individual strangeness value to/lfor each said training classified 
example (i~L2...A and said at least one unclassified example fc=/±lL having an allocated 
potential classification!^, and a single Tan overall] strangeness value (d(v)) valid under 

4 
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*e indettffld^^ assumption for each said classification 

set in dependence on the individual strangeness value s (a.) of each flcamole bvflyp 
formula 

dM J{i^J . wheref=1 , 2 _ 1M: 

selecting the said classification set to which the most likely allocated potential 
classification for said at least one unclassified example belongs, wherein said predicted 
classification is the most likely allocated potential classification in dependence on said 
| &in&Le. roverall1 strangeness values; 

determining a confidence value for said predicted classification on the basis of the 
| gjngl^foverall] strangeness value assigned to one of said classification sets to which the 
second most likely allocated potential classification for said at least one unclassified 
example belongs; and 

outputting said predicted classification for said at least one unclassified example 
and said confidence value for said predicted classification, 

16. (Canceled) 

| 17. A computer-itrmlementeff data classification method as claimed in claim 15, 
wherein said selected classification set is selected without the application of any general 
rules determined from the said training set. 

18. A data carrier on which is stored a classification program for classifying data by 
performing the following steps: 

generating a plurality of classification sets, each said classification set containing a 
plurality J£U>f training classified examples with their classification and at least one 
unclassified example (7+1 ) that has been allocated a potential classification; 

determining an individual strangeness value (a^fot each said training classified 
example (z'=L2.. Jl and said at least one unclassified example 0=/+^ h aving an allocated 
potential classification^, and a^ingl£_[an overall] strangeness value fdfvY) valid under 
the [iid] independent ly and identical distt ihufecL assumption for each said classification 
set in dependence on said individual strangeness value sfaA of each example bv the 
formula 

d(y) - ft''***'*! . where f=1.2. .././+l: 

selecting the classification set to which the most likely allocated potential 
classification for the said at least one unclassified example belongs, wherein the predicted 
classification is the most likely allocated potential classification in dependence on said 
SiaglaJpverall] strangeness values; and 

determining a confidence value for said predicted classification on the basis of said giagk 
[overall] strangeness value assigned to one of said classification sets to which the second 
most likely allocated potential classification for said at least one unclassified example 
belongs. 
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REMARKS 

Applicant files herewith an amended set of claims, which have been amended to 
overcome the Examiner's objections- 
Claims 10, 12 and 14 have been amended to recite "A data classification 
apparatus...". Claims 10, 14, 15 and 18 have been amended to ensure that no new matter 
is added In these claims, the phrase "an overall strangeness value" has been replaced by 
the phrase "a single strangeness value", in accordance with page 12, line 10. Claims 15 
and 17 now recite "a computer-implemented data classification method" to ensure that 
they relate to statutory subject matter. 

The claims have also been amended to ensure that they are fully distinguished 
from the prior art, as explained below. In particular, claims 10, 14, 15 and 18 now 
include the formula by which the single strangeness value is determined by the assay 
means. This formula is not shown or suggested in any of the prior art documents. 

The invention relates to an apparatus for and a method of data classification of 
unknown items, using a training set of classified examples. 

The aim of the invention is not only to predict a classification for an unknown 
item, but also to provide a measure of confidence in that classification, valid under the iid 
(independently and identically distributed) assumption. This is the assumption that the 
training and unknown examples are generated from the same distribution. 

As defined in amended claim 10, the data classification apparatus comprises an 
input device for receiving training classified examples and at least one unclassified 
example, a memory for storing the examples, an output terminal for outputting a 
predicted classification for the unclassified example and a processor for identifying the 
predicted classification, where the processor includes classification allocation means for . 
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allocating potential classifications to each unclassified example and for generating a 
plurality of classification sets, each set containing the plurality of training classified 
examples (/) with the classification and at least one unclassified example (/+l) with its 
allocated potential classification, an assay means including an example valuation device 
which determines individual strangeness values (a,) for each training classified example 
0=1,2.../) and at least one unclassified example (f=/+1) having an allocated potential 
classification (y), the assay means determining a single strangeness value (d(y)) valid 
under the iid assumption for each classification set in dependence on the individual 
strangeness values (a f ) of each example by a given formula, a comparative device for 
selecting the classification set to which the most likely potential classification for the 
unclassified example belongs on the basis of the single strangeness value assigned by the 
assay means, and a strength of prediction monitoring device for determining a confidence 
value for the predicted classification on the basis of the single strangeness value assigned 
to the classification set to which the second most likely potential classification of the 
unclassified example belongs. 

\{i w a >a H 

The formula d{y) = a ' ' ~ M \ where i=l ,2 ..../, /+1 , calculates the 

/"hi 

single strangeness value by counting how many of the individual strangeness values of 
the training examples are greater than or equal to the individual strangeness value of the 
unclassified example, and dividing this by the total number of examples. 

Thus, the apparatus starts with training classified examples, and uses these to 
predict the classification of an unclassified example. The classification allocation mean 
allocates potential classifications to each unclassified example, and generates a number of 
classification sets, each containing the said training examples with their classification, 
and at least one unclassified example with a potential classification. Each pair of the 
example and classification in the set is tested by the assay means to determine its 
individual strangeness value. That is a measure of how strange it would be if the 
classification were correct. The assay means then determines a single strangeness value, 
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valid under the iid assumption, for each classification set, in dependence on the individual 
strangeness values. The comparative device compares the single strangeness value for all 
the sets, to see which is most likely to be correct for the unclassified example. The 
strength of prediction monitoring device then looks at the second most likely 
classification, on the basis of the single strangeness values, to give a measure of 
confidence that the most likely classification is in fact correct. Thus, the greater the 
difference between the single strangeness values of the most likely potential classification 
and the second most likely potential classification, the greater is the measure of 
confidence that the most likely classification is correct. 

We submit that this combination of features, and in particular the formula, is not 
shown in any of the prior art documents. 

Applicant agrees that Mizuno shows an input device, a memory, an output 
terminal and a processor, with the processor including classification allocation means and 
assay means. However, applicant submits that Mizuno calculates an error between a 
classification assigned to an unknown pattern and a correct classification, and initiates re- 
training when the average error exceeds a predetermined value. This copes with the 
possibility that the unknown patterns are generated by a different mechanism from the 
training patterns (see column 1, lines 9 to 19). This differs from the invention as claimed, 
which specifies that the single strangeness value is valid under the iid assumption, in that 
the unknown and training examples are generated from the same distribution. It is 
submitted that column 7, lines 1 to 12 of Mizuno do not show the determination of an 
individual strangeness value for each training example and an unknown example and then 
the determination of a single strangeness value for each classification set, in dependence 
on the individual strangeness values. In particular, Mizuno does not show the formula 
now included in amended claim 10. Applicant agrees that Mizuno does not teach a 
strength of prediction monitoring device for determining a confidence value for the most 
likely classification, on the basis of the single strangeness value of the second most likely 
potential classification. 
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It is submitted therefore that claim 10 is clearly novel over Mizuno. Further, 
Mizuno does not show or suggest determining a single strangeness value valid under the 
iid assumption, and indeed is concerned with the possibility that the unknown patterns are 
generated from a different distribution, rather than the same distribution. Thus, Mizuno 
teaches away from the invention, rather than towards it In addition, Mizuno does not 
show or suggest determining individual strangeness values and then a single strangeness 
value, either in general or on the basis of the claimed formula, nor the strength of 
prediction monitoring device for a confidence value on the basis of the single strangeness 
value of the second most likely potential classification, 

Tsuboka relates to pattern recognition employing the Hidden Markov Model, and 
is particularly relevant to speech recognition or other time series. In such pattern 
recognition it is necessary to identify a spoken word, and to decide which of a number of 
classified words it is most likely to be. Examples of classified words are put into clusters 
and then the probability is calculated that the word belongs to a particular cluster. 
Tsuboka recognises that a pattern to be recognised has different probabilities of belonging 
to different clusters, and calculates the probability in various ways, using a comparative 
device to select the most likely classification. However, the aim of Tsuboka is simply to 
find the cluster with the highest probability, for accuracy of recognition, 

Applicant submits that firstly Tsuboka does not use the iid assumption, as claimed 
by the applicant. Secondly, applicant submits that Tsuboka does not use the second most 
likely classification as a strength of prediction monitoring device. The passage indicated 
by the Examiner (column 1 1 lines 1 1 to 20) appears simply to say that a pattern could be 
in more than one classification with different probabilities. The subsequent portion of the 
specification indicates how the most likely classification is then computed. Tsuboka 
clearly does not show the use of the formula, as now included in amended claim 10. 

Thus, it is submitted that claim 10 is clearly novel over Tsuboka. Further, 
Tsuboka does not show or suggest using the second most likely allocated potential 
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classification as a strength of prediction monitoring device for determining a confidence 
value of the predicted classification. Claim 10 is therefore clearly inventive in view of 
Mizuno. 

With regards to a combination of Mizuno and Tsuboka, it is submitted that they 
would not lead to the present invention. Firstly, neither is concerned with ensuring that 
the strangeness value is valid under the iid assumption, and indeed Muuno is concerned 
with the possibility that the unknown patterns are generated from a different distribution. 
In addition, neither Mizuno nor Tsuboka shows the use of the second most likely 
classification as the strength of prediction monitoring device to provide a confidence 
value for the correctness of the predicted classification. 

Thus, a combination of Mizuno and Tsuboka, even though they are both 
concerned with maintaining accuracy of the classification, simply would not lead to the 
present invention, as claimed in claim 10, because of the lack of provision of a confidence 
value using the second most likely classification, and the fact that the strangeness values 
are not valid under the iid assumption. The combination also fails to show the use of the 
formula enumerated in the claim. 

Applicant submits therefore that claim 10 is clearly distinguished from all the prior 
art, and is therefore allowable. Claims 14, 15 and 18 have the same limitations as claim 
10, and it is therefore submitted that these are also allowable. 

As claims 12 and 17 are dependent on claims 10 and 15, it is submitted that these 
are allowable to the same extent. 
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Conclusion 

The application is considered in good and proper form for allowance, and the 
Examiner is respectfully requested to pass this application to issue. If, in the opinion of the 
Examiner, a telephone conference would expedite the prosecution of the subject application 
the Examiner is invited to call the undersigned attorney. 



Respectfully submitted, 




Pippenger, Keg. No. 46,055 
LEYDIG, VOIT & MAYER, LTD. 
Two Prudential Plaza, Suite 4900 
1 80 North Stetson Avenue 
Chicago, Illinois 60601-6780 
(312) 616-5600 (telephone) 

Date: November 18, 2004 ^ 6l6 " 57 °° (fecsimi,€ > 

Amendment or ROA - Fund (Revised 10/21/2004) 



11 

iflflEJA/u* mm st u/mrcnni mm« dm fPacfom ghnrbrri tm* sw-ii!S0Tfl.ccypw!9(i • iptifflffiMI • WW fi16 5700' DURATION fmm-ssM-O 1 



